Graceful restart (GR) methods and devices

ABSTRACT

This disclosure provides methods and devices for Graceful Restart (GR) in a network with multiple GR Restarters that include a first GR Restarter and at least one second GR Restarter corresponding to the first GR Restarter. After receiving routes from all normal neighbor network devices of the first GR Restarter, the first GR Restarter performs route selection to determine first selected routes, and sends the first selected routes to the at least one second GR Restarter. After receiving routes from the at least one second GR Restarter, the first GR Restarter performs route selection to determine second selected routes, updating a local Forwarding Information Base (FIB) at the first GR Restarter with the second selected routes, and sends the second selected routes to all neighbor network devices of the first GR Restarter. The first GR Restarter directly connects with the at least one second GR Restarter corresponding to the first GR Restarter.

The present application is a 371 application of InternationalApplication No. PCT/CN2012/087261 filed on Dec. 24, 2012 and entitled“Graceful Restart (GR) Methods and Devices,” which claims the benefit ofChinese Patent Application No. 201110455301.X filed on Dec. 28, 2011.

BACKGROUND

In an application field with relatively high reliability requirements,when a host routing engine configured on a network device becomesabnormal or breaks down, a Graceful Restart (GR) function is used toensure uninterrupted traffic forwarding. Two kinds of network devices,i.e. a GR Restarter and a GR Helper, used in the GR process are definedin the GR protocol. A failed network device which is restarted is calledthe GR Restarter, and a neighbor network device of the GR Restarter iscalled a GR Helper. The GR Restarter executes the GR function with thehelp of the GR Helper.

When a network device is restarted, the network device acting as the GRRestarter sends an open packet to all neighbor network devices acting asthe GR Helpers to re-establish a connection, and performs routeselection after receiving routes sent from all the neighbor networkdevices.

BRIEF DESCRIPTION OF THE DRAWINGS

Features of the present disclosure are illustrated by way ofnon-limiting examples in the following figure(s), in which:

FIG. 1 is a schematic diagram illustrating structure of an examplenetwork based on a BGP protocol;

FIG. 2 is a flow chart illustrating a BGP GR method in accordance withan example of the present disclosure;

FIG. 3 is a schematic diagram illustrating a GR process of two GRRestarters in the network shown in FIG. 1 in accordance with an exampleof the present disclosure;

FIG. 4 is a schematic diagram illustrating a GR process of three GRRestarters in the network shown in FIG. 1 in accordance with an exampleof the present disclosure;

FIG. 5 is a schematic diagram illustrating a GR process of four GRRestarters in the network shown in FIG. 1 in accordance with an exampleof the present disclosure;

FIG. 6 is a schematic diagram illustrating another GR process of four GRRestarters in the network shown in FIG. 1 in accordance with an exampleof the present disclosure;

FIG. 7 is a schematic diagram illustrating structure of a GR Restarterin accordance with a first example of the present disclosure; and

FIG. 8 is a schematic diagram illustrating structure of a GR Restarterin accordance with a second example of the present disclosure.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the present disclosure isdescribed by referring to non-limiting examples. In the followingdescription, numerous specific details are set forth in order to providea thorough understanding of the present disclosure. It will be readilyapparent however, that the present disclosure may be practiced withoutlimitation to these specific details. In some instances, some methodsand structures have not been described in detail so as not tounnecessarily obscure the present disclosure. As used throughout thepresent disclosure, the term “includes” means includes but not limitedto, the term “including” means including but not limited to. The term“based on” means based at least in part on. In addition, the terms “a”and “an” are intended to denote at least one of a particular element.

Graceful Restart is useful in many routing protocols such as BorderGateway Protocol (BGP), Open Shortest Path First (OSPF), andIntermediate System To Intermediate System (IS-IS) etc. BGP is a routingprotocol for dynamically exchanging routing information betweenautonomous systems. The BGP is taken as external routing protocolstandard of the Internet, and is widely used by the Internet ServiceProvider (ISP). In the following non-limiting examples, a GR process forBGP is used but it will be appreciated that other suitable routingprotocols may also be used.

When the BGP of multiple neighbor network devices is restarted, the GRprocess may become infeasible because a deadlock situation might arisewhen two neighboring GR Restarters waits for routes from each other inorder to perform route selection during the GR process. One example BGPGR process of the multiple neighbor network devices is described withreference to FIG. 1.

FIG. 1 is a schematic diagram illustrating structure of an examplenetwork for implementing a BGP protocol. The network includes variousnetwork devices: a Provider Edge (PE) 1, a PE2, a PE3, a Customer Edge(CE) 1 and a CE2. It will be appreciated that the devices in the networkmay be any routers, and switches etc. The PE1, PE2 and PE3 interconnectwith each other. The CE1 connects with the PE1 and the CE2 connects withthe PE2. Suppose that the BGP of the PE1 is restarted, and neighbornetwork devices PE2, PE3 and CE1 are normal, the PE1 is the GR Restarterand the PE2, PE3 and CE1 are GR Helpers. After the BGP of the PE1 isrestarted, the PE1 respectively establishes the BGP connection with thePE2, PE3 and CE1. The PE2, PE3 and CE1 respectively send a route orrouting information to the PE1. After receiving the routes sent from thePE2, PE3 and CE1, the PE1 performs route selection, updates a localForwarding Information Base (FIB) with the result of the routeselection. Then, the GR process ends.

However, if the BGP of both the PE1 and PE2 is restarted, both the PE1and PE2 are the GR Restarters. The PE2, PE3 and CE1 are the GR Helpersof the PE1. The PE3 and CE1 are in a normal working state, and sendroutes to the PE1. Since the BGP of the PE2 is also restarted, the PE2does not send the route to the PE1. The PE1, PE3 and CE2 are the GRHelpers of the PE2. The PE3 and CE2 are in a normal working state, andsend the routes to the PE2. Since the BGP of the PE1 is restarted, thePE1 does not send a route to the PE2. Therefore, the PE1 and PE2 waitfor a route from each other. In fact, the PE1 and PE2 cannot receive anEnd-Of-Rib (EOR) message that represents the completion of the initialrouting update from each other, which results in deadlock.

In order to prevent or reduce the likelihood of the above deadlockproblem, Request for Comments (RFC) 4724 provides that when the BGP of anetwork device is restarted, and a restart flag R needs to be configuredin an Open packet. After receiving an Open packet carrying the restartflag R from a neighbor network device, the network device will not waitfor the route sent from the neighbor network device. Although thenetwork device will not wait for the route sent from the neighbornetwork device when its BGP restarts, the network device will delete theroute previously received from the neighbor network device before BGP GRis performed, which results in interruption of traffic.

Referring to FIG. 2, which is a flow chart illustrating a GR methodsuitable for BGP in accordance with an example of the presentdisclosure. The method includes the following blocks.

Block 201: When there are multiple GR Restarters, the multiple GRRestarters include at least one first (“master”) GR Restarter and asecond (“slave”) GR starter corresponding to each master GR Restarter.The master GR Restarter and slave GR Restarter may be selected from themultiple GR Restarters, for instance, they may be selected prior to therestarting process. As will be shown in the examples in FIGS. 3 to 6,each pair of master GR Restarter and slave GR Restarter in the networkare directly connected with each other. Throughout the presentdisclosure, the term “directly connects” refers to a connection over thenetwork layer (L3) of the Internet protocol suite. For example, thedirect connection may be a Transmission Control Protocol (TCP)connection etc. The term “neighbor network devices” refers to networkdevices that are directly connected with each other over the networklayer. However, network devices with direct connection over the networklayer may be connected via other devices over the physical layer.

The multiple GR Restarters may form an aggregation, in which each GRRestarter in the aggregation directly connects with at least one otherGR Restarter in the aggregation.

In this block, the method for selecting the at least one master GRRestarter and a slave GR Restarter corresponding to each master GRRestarter from the multiple GR Restarters includes: selecting, as themaster GR Restarter, one GR Restarter from any two directly-connected GRRestarters in the multiple GR Restarters according to a pre-set rule,and taking the other GR Restarter as the slave GR Restarter of themaster GR Restarter. The selection may be based on the identifier (ID)of the GR Restarters. For example, the pre-set rule may include:selecting the GR Restarter with the maximum or minimum ID from the twodirectly-connected GR Restarters as the master GR Restarter. The ID ofeach GR Restarter may uniquely identify the GR Restarter, and isdifferent from the ID of any other GR Restarter. The twodirectly-connected GR Restarters may send packets carrying their IDs toeach other, by which one party may obtain the ID of the other party. Theselection of the master GR Restarter may be performed by the master orslave GR Restarter prior to the GR process. A master GR Restarter has atleast one slave GR Restarters, and a slave GR Restarters has at leastone master GR Restarter.

Block 202: From the master GR Restarter's perspective, after the masterGR Restarter has received routes sent from all normal neighbor networkdevices of the master GR Restarter, route selection is performed todetermine “first selected routes”, and the first selected routes aresent to all the slave GR Restarter(s) of the master GR Restarter. If themaster GR Restarter receives selected routes sent from all of its slaveGR Restarter(s), route selection is performed to determine secondselected routes, a local FIB at the master GR Restarter is updated withthe second selected routes, and the second selected routes are sent toall neighbor network devices.

Before calculating the first selected routes, the master GR Restarterneeds not wait for the routes sent from its slave GR Restarter(s). Aslong as routes are received from all the normal neighbor networkdevices, the master GR Restarter is able to perform route selection todetermine the first selected routes, and sends the first selected routesto its slave GR Restarter(s). As such, the deadlock problem resultingfrom the GR Restarter and its neighbor GR Restarter waiting for theroute sent from each other can be avoided. In the present disclosure,the “normal” neighbor network devices refer to the devices that are innormal working state and need not to be restarted.

Block 203: After the slave GR Restarter received routes from all of itsneighbor network devices, the slave GR Restarter performs routeselection to determine third selected routes.

If complete routes sent from all of its neighbor network devices arereceived, the local FIB is updated with the third selected routes;

otherwise, the third selected routes are sent to its master GRRestarter, route selection is performed to determine fourth selectedroutes until complete routes sent from all the neighbor network devicesare received, the local FIB at the slave GR Restarter is updated withthe fourth selected routes, and the fourth selected routes are sent toall the neighbor network devices.

Although the terms ‘first’, ‘second’, ‘third’ and ‘fourth’ have beenused throughout the disclosure to refer to ‘selected routes’, it shouldbe understood that they are merely labels for easy reference, and notintended to represent any specific sequence or order unless indicatedotherwise. For example, in the case of ‘third’ and ‘fourth’ selectedroutes, the terms ‘third’ and ‘fourth’ are merely used to distinguishthem from the ‘first’ and ‘second’ selected routes determined by themaster (first) GR restarter.

The slave GR Restarter performs route selection to determine the thirdselected routes after receiving the routes sent from all the neighbornetwork devices including the master GR Restarter.

In practice, the local FIB is updated with the second selected routesafter the master GR Restarter receives all the complete routes, and thesecond selected routes are sent to all the neighbor network devices.Before receiving all the complete routes, the master GR Restarter needsto exchange route information with its slave GR Restarter(s) one or moretimes, until all the complete routes are received. Similarly, the localFIB at the slave GR Restarter is updated with the fourth selected routesafter the slave GR Restarter receives all the complete routes, and thefourth selected routes are sent to all the neighbor network devices.Before all the complete routes are received, the slave GR Restarterneeds to exchange route information with its master GR Restarter(s) oneor more times, until all the complete routes are received. Therefore,after the first selected routes are determined by the master GRRestarter, the method further includes: it is determined whether thecomplete routes sent from all the neighbor network devices are received.If yes, the local FIB is updated with the first selected routes;otherwise, the master GR Restarter continues to exchange routeinformation with either all of its slave GR Restarter(s), or those whichdo not send the complete routes.

If the master GR Restarter continues to exchange route information withall of its slave GR Restarter(s), it needs to send the first selectedroutes to all of its slave GR Restarter(s). After sending the firstselected routes to all of its slave GR Restarter(s), if the master GRRestarter receives the routes sent from all of its slave GRRestarter(s), the master GR Restarter needs to calculate the secondselected routes, and determines whether all the completes routes arereceived. If yes (complete), the local FIB is updated with the secondselected routes; otherwise (incomplete), the second selected routes aresent to all of its slave GR Restarter(s).

In this situation, the method for sending, by the master GR Restarter,the first selected routes to all of its slave GR Restarters includes:

if the master GR Restarter receives complete routes sent from all normalneighbor network devices of the master GR Restarter, but does notreceive complete routes sent from its at least two (i.e. more than one)slave GR Restarters, denoting the first selected routes as incompleteroutes, and sending the first selected routes denoted as incompleteroutes to all the slave GR Restarter(s) of the master GR Restarter;

if the master GR Restarter receives complete routes sent from all thenormal neighbor network devices of the master GR Restarter, but does notreceive a complete route sent from at most one slave GR Restarter,denoting the first selected routes as complete routes, and sending thefirst selected routes denoted as complete routes to the at most oneslave GR Restarter, denoting the first selected routes as incompleteroutes, and sending the first selected routes denoted as incompleteroutes to all the other slave GR Restarter(s), if any;

if the master GR Restarter does not receive the complete routes sentfrom all the normal neighbor network devices of the master GR Restarter,denoting the first selected routes as incomplete routes, and sending thefirst selected routes denoted as incomplete routes to all of its slaveGR Restarters.

To summarise, if routes are not received from all normal neighbornetwork devices, the received routes are incomplete. If routes arereceived from all normal neighbor network devices, but not from morethan one slave GR Restarter, the received routes are also incomplete.However, If routes are received from all normal neighbor networkdevices, but not from at most one slave GR Restarter, the receivedroutes are complete.

A slave (second) GR Restarter “aggregation” may be used to represent theslave GR Restarter(s) that have not sent complete routes to the masterGR Restarter. If the master GR Restarter exchanges route informationwith all the slave GR Restarter(s) which do not send complete routes,the master GR Restarter determines a slave GR Restarter aggregation,sends the first selected routes to all the slave GR Restarter(s) in theGR Restarter aggregation. After the master GR Restarter sends the firstselected routes to all slave GR Restarter(s) in the slave GR Restarteraggregation, if the master GR Restarter receives routes sent from allthe slave GR Restarter(s) in the slave GR Restarter aggregation, themaster GR Restarter calculates the second selected routes, anddetermines whether all the complete routes are received. If completeroutes are received, the local FIB is updated with the second selectedroutes; otherwise, the slave GR Restarter aggregation is determinedagain. The method for determining the slave GR Restarter aggregation mayinclude: for each slave GR Restarter of the master GR Restarter, if acomplete route of a slave GR Restarter is not received, the slave GRRestarter is added to the slave GR Restarter aggregation.

In this situation, the method for sending the first selected routes toall the slave GR Restarter(s) in the slave GR Restarter aggregation ofthe master GR Restarter may include:

if the master GR Restarter receives complete routes sent from all normalneighbor network devices of the master GR Restarter, and there are atleast two GR Restarters in the slave GR Restarter aggregation, denotingthe first selected routes as incomplete routes, and sending the firstselected routes denoted as incomplete routes to all the slave GRRestarters in the slave GR Restarter aggregation;

if the master GR Restarter receives complete routes sent from all thenormal neighbor network devices of the master GR Restarter, and there isat most one slave GR Restarter in the slave GR Restarter aggregation,denoting the first selected routes as complete routes, and sending thefirst selected routes denoted as the complete routes to the at most oneslave GR Restarter;

if the master GR Restarter does not receive the complete routes sentfrom all the normal neighbor network devices, denoting the firstselected routes as the incomplete routes, and sending the first selectedroutes denoted as the incomplete routes to all the slave GR Restarter(s)in the GR Restarter aggregation.

Similarly, after the slave GR Restarter performs route selection todetermine the third selected routes, and before the local FIB is updatedwith the third selected routes, the method further includes:determining, by the slave GR Restarter, whether complete routes of allthe neighbor network devices are received, and if yes, updating thelocal FIB with the third selected routes; otherwise, the slave GRRestarter continues to exchange route information with all of its masterGR Restarter(s), or exchange route information with all master GRRestarter(s) which do not send the complete routes.

If the slave GR Restarter continues to exchange route information withall of its master GR Restarter(s), it needs to send the third selectedroutes to all of its master GR Restarter(s), calculates the fourthselected routes until the complete routes sent from all the neighbornetwork devices are received, updates the local FIB at the slave GRRestarter with the fourth selected routes, and sends the fourth selectedroutes to all the neighbor network devices.

In this situation, the method for sending the third selected routes toall of its master GR Restarters includes:

if the slave GR Restarter receives complete routes sent from all normalneighbor network devices, and if an incomplete route sent from one ofits master GR Restarter(s) is not received, or if complete routes sentfrom all of its master GR Restarter(s) are received, but complete routessent from at least two slave GR Restarter(s) are not received, denotingthe third selected routes as incomplete routes, and sending the thirdselected routes denoted as incomplete routes to all of its master GRRestarter(s) and any other GR Restarter(s);

if the slave GR Restarter receives the complete routes sent from all thenormal neighbor network devices, and if a complete route sent from itsat most one slave GR Restarter is not received, denoting the thirdselected routes as the complete routes, and sending the third selectedroutes denoted as complete routes to the at most one slave GR Restarter,denoting the third selected routes as incomplete route, and sending thethird selected routes denoted as incomplete routes to all of its masterGR Restarter(s) and any other GR Restarter(s).

In the above, a master (first) GR Restarter “aggregation” may be used torepresent the master GR Restarter(s) that have not sent complete routesto the slave GR Restarter. In this case, if the slave GR Restartercontinues to exchange route information with all the master GRRestarter(s) which do not send the complete routes, the slave GRRestarter needs to determine a master GR Restarter aggregation, andsends the third selected routes to all the master GR Restarter(s) in themaster GR Restarter aggregation, performs route selection to determinethe fourth selected routes until complete routes sent from all of itsneighbor network devices are received, updates the local FIB at theslave GR Restarter with the fourth selected routes, and sends the fourthselected routes to all of its neighbor network devices. The master GRRestarter aggregation may be determined as follows: for each of itsmaster GR Restarter(s), if a complete route sent from the master GRRestarter is not received, adding the master GR Restarter to the masterGR Restarter aggregation.

In this situation, the method for sending the third selected routes toall the master GR Restarter(s) in the master GR Restarter aggregationmay include:

if the slave GR Restarter receives complete routes sent from allneighbor network devices, and if an incomplete route sent from one ofits master GR Restarters is not received, denoting the third selectedroutes as incomplete routes, and sending the third selected routesdenoted as the incomplete routes to the master GR Restarter;

if the slave GR Restarter receives the complete routes sent from all thenormal neighbor network devices, and if a complete route sent from itsat most one slave GR Restarter is not received, denoting the thirdselected routes as complete routes, and sending the third selectedroutes denoted as complete routes to the at most one slave GR Restarter,denoting the third selected routes as incomplete routes, and sending thethird selected routes denoted as incomplete routes to all the master GRRestarter(s) in the master GR Restarter aggregation.

In the conventional method, after sending a route to its neighbornetwork devices, a network device needs to send an End-of-Rib (EOR)message to notify its neighbor network devices that the transmission ofthe route is completed. After receiving the EOR message sent from thenetwork device, the neighbor network device determines that completeroutes of the network device are received.

In the example of the present disclosure shown in FIG. 2, in order todetermine whether the current GR Restarter has received all completeroutes, the EOR messages are divided as two kinds of messages: onecarries a Partial flag and the other does not (The EOR message in theconventional method does not carry the Partial flag).

When an EOR message carrying the Partial flag is sent, it denotes thatthe routes sent are selected routes determined in a situation where,other than a target network device receiving the EOR message, thecomplete routes sent from at least one neighbor network device are notreceived. For instance, a device A sends a route to a device B (i.e.,the target network device), and sends an EOR message carrying thePartial flag, which denotes that at present, device A has not receivecomplete routes sent from at least one neighbor network device otherthan device B.

When an EOR message which does not carrying the Partial flag is sent, itdenotes that the routes sent are selected routes determined in asituation where, other than the target network device, complete routessent from all other neighbor network devices are received. For instance,the device A sends the route to the device B (target network device),and sends the EOR message which does not carry the Partial flag, whichdenotes that the device A receives the complete routes sent from all theneighbor network devices except for the device B.

Therefore, the method for denoting the first selected routes as completeroutes includes: generating a first EOR message which does not carry thePartial flag. The method for sending the first selected routes which aredenoted as complete routes to the slave GR Restarter includes: sendingthe first selected routes to the slave GR Restarter, and sending thefirst EOR message which does not carry the Partial flag to the slave GRRestarter.

The method for denoting the third selected routes as complete routesincludes: generating a third EOR message which does not carry thePartial flag. The method for sending the third selected routes denotedas complete routes to the master GR Restarter includes: sending thethird selected routes to the master GR Restarter, and sending the thirdEOR message which does not carry the Partial flag to the master GRRestarter.

The method for denoting the first selected routes as incomplete routesincludes: generating a second EOR message carrying the Partial flag. Themethod for sending the first selected routes which are denoted asincomplete routes to the slave GR Restarter includes: sending the firstselected routes to the slave GR Restarter, and sending the second EORmessage carrying the Partial flag to the slave GR Restarter.

The method for denoting the third selected routes as incomplete routesincludes: generating a fourth EOR message carrying the Partial flag. Themethod for sending the third selected routes to the master GR Restarterincludes: sending the third selected routes to the master GR Restarter,and sending the EOR message carrying the Partial flag to the master GRRestarter.

The example BGP GR method shown in FIG. 2 is described hereafter withreference to FIGS. 3 to 6.

In the following example in FIG. 3, the BGP of two network devices isrestarted. The GR process for restarting the BGP of the two networkdevices is described as follows.

Suppose that the BGP of network devices, i.e. PE1 and PE2 in FIG. 1 isrestarted. The network devices, i.e. PE1 and PE2 are the GR Restarters.The PE1 and PE2 are sorted according to a descending order of the ID,and the sort result is PE1 and PE2 (i.e. PE1's ID is greater of thetwo).

According to the method in the example of the present disclosure shownin FIG. 2, one of the two directly-connected GR Restarters with themaximum ID is taken as the master GR Restarter. The GR process of PE1and PE2 is shown in FIG. 3. FIG. 3 is a schematic diagram illustratingan example GR process of two GR Restarters in a TRILL (TRansparentInterconnection of Lots of Links) network shown in FIG. 1.

A master GR Restarter and a slave GR Restarter are selected from PE1 andPE2. The PE1 is selected as the master GR Restarter of the PE2, and thePE2 is selected as the slave GR Restarter of the PE1.

The PE1, acting as the master GR Restarter (“first GR Restarter”) of thePE2, performs route selection to determine the selected routes afterreceiving routes sent from all the neighbor network devices, i.e. CE1and PE3 except for its slave GR Restarter, i.e. the PE2. Since the PE1receives complete routes from all the neighbor network devices exceptfor the PE2, the selected routes are denoted as the complete routes (TheRoute+EOR is used for denoting that the sent routes are complete routesin FIGS. 3 to 5), and sent to the PE2. That is, the selected routes aresent to the PE2, and the EOR message which does not carry the Partialflag is sent to the PE2. After sending the routes, the CE1 and PE3, asGR Helpers in a normal working state, send the EOR message, which doesnot carry the Partial flag, to the PE1. It should be noted that in thefollowing examples, after the GR Helpers in the normal working statesends the routes, the EOR message sent from the GR Helpers does notcarry the Partial flag.

The PE2, acting as the slave GR Restarter (“second GR Restarter”) of thePE1, performs route selection to determine the selected routes afterreceiving the routes sent from all the neighbor network devices, i.e.PE1, CE2 and PE3. Since the EOR messages sent from the PE1, CE2 and PE3do not carry the Partial flag, it may be determined that all completeroutes are received. The local FIB is updated with the selected routes,and the selected routes and the EOR message which does not carry thePartial flag are sent to all the neighbor network devices, i.e. PE1, CE2and PE3, and the PE2 finishes the GR process.

After receiving the selected routes and the EOR message which does notcarry the Partial flag sent from the PE2, the PE1 calculates theselected routes, and determines that all the complete routes arereceived. The local FIB is updated with the selected routes, theselected routes and the EOR message which does not carry the Partialflag are sent to all the neighbor network device, i.e. CE1, PE2 and PE3,and the PE1 finishes the GR process.

The PE1 and PE2 finish the GR process after receiving all the completeroutes. The problem of traffic interruption can be avoided.

In the following example in FIG. 4, the BGP of three network devices isrestarted. The GR process for restarting the BGP of the three networkdevices is as follows (taking route information exchange with all of itsslave GR Restarter(s) or master GR Restarter(s) which do not send thecomplete routes for example).

Suppose that the BGP protocol of the network devices, i.e. CE1, PE1 andPE2 is restarted. The CE1, PE1 and PE2 are the GR Restarters. The CE1,PE1 and PE2 are sorted according to a descending order of the ID, andthe sort result is PE1, PE2 and CE1.

According to the method of the example in the present disclosure shownin FIG. 2, one of each pair of two directly-connected GR Restarters withthe maximum ID is taken as the master GR Restarter. The GR process ofCE1, PE1 and PE2 is shown in FIG. 4. FIG. 4 is a schematic diagramillustrating a GR process of three GR Restarters in the network shown inFIG. 1 in accordance with an example of the present disclosure.

The master GR Restarters and slave GR Restarters are selected from anytwo directly-connected GR Restarters in the CE1, PE1 and PE2. The CE1directly connects with the PE1. Since the ID of the PE1 is larger thanthat of the CE1, the PE1 is selected as the master GR Restarter of theCE1, and the CE1 is selected as the slave GR Restarter of the PE1. ThePE1 directly connects with the PE2. Since the ID of the PE1 is largerthan that of PE2, the PE1 is selected as the master GR Restarter of thePE2, and the PE2 is selected as the slave GR Restarter of the PE1.

The PE1 is selected as the master GR Restarter of the CE1 and PE2. Afterreceiving the routes sent from the normal neighbor network device, i.e.the PE3 but not from its slave GR Restarters, i.e. CE1 and PE2, the PE1calculates the selected routes, determines that a slave GR Restarteraggregation is {CE1, PE2}. Since the complete routes sent from at leasttwo slave GR Restarters CE1 and PE2 are not received, the selectedroutes are denoted as the incomplete routes (In FIGS. 4 and 5, theRoute+EOR (P) denotes that the sent routes are the incomplete routes),and sent to all the slave GR Restarters of the PE1, i.e. CE1 and PE2.

The CE1 is selected as the slave GR Restarter of the PE1. Afterreceiving incomplete routes sent from the PE1, the CE1 performs routeselection to determine the selected routes, and determines that a masterGR Restarter aggregation is {PE1}. Since there is no other neighbornetwork device besides the PE1, it may be considered that completeroutes of all the neighbor network devices except for the PE1 arereceived. Therefore, the selected routes are denoted as complete routes,and sent to the PE1.

The PE2 is selected as the slave GR Restarter of the PE1. Afterreceiving routes from the PE3 and CE2, and incomplete routes from thePE1, the PE2 performs route selection to determine the selected routes,and determines that the master GR Restarter aggregation is {PE1}. Sincecomplete routes sent from all the neighbor network devices except forthe PE1 are received, the selected routes are denoted as completeroutes, and sent to the PE1.

After receiving the routes denoted as complete routes sent from the CE1and PE2, the PE1 performs route selection to determine the selectedroutes. Since the complete routes are received, the local FIB is updatedwith the selected routes, and the selected routes are denoted ascomplete routes, and sent to all the neighbor network devices, i.e. CE1,PE2 and PE3, and the PE1 finishes the GR process.

After receiving routes which are denoted as complete routes and sentfrom the PE1, CE1 performs route selection to determine the selectedroutes. Since all complete routes are received, the local FIB is updatedwith the selected routes, and the selected routes are denoted ascomplete routes, and sent to CE1's neighbor network device, i.e. PE1,and the CE1 finishes the GR process.

After receiving the routes denoted as complete routes from the PE1, PE2performs route selection to determine the selected routes. Since allcomplete routes are received, the local FIB is updated with the selectedroutes, which are denoted as the complete routes, and sent to all ofPE2's neighbor network devices, i.e. the PE1, PE3 and CE2, and the PE2finishes the GR process.

The CE1, PE1 and PE2 finish the GR process. Since each GR Restarterfinishes the GR process after receiving all the complete routes, theproblem of traffic interruption can be avoided.

In the following example in FIG. 5, the BGP of four network devices isrestarted. The GR process for restarting the BGP of the four networkdevices is as follows (taking the route information exchange with all ofits slave GR Restarter(s) or master GR Restarter(s) which do not sendthe complete routes for example).

Suppose that the BGP protocol of the network devices, i.e. CE1, PE1, PE2and CE2 shown in FIG. 1 is restarted. The CE1, PE1, PE2 and CE2 are theGR Restarters. The CE1, PE1, PE2 and CE2 are sorted according to adescending order of the ID, and the sort result is PE1, CE2, PE2 andCE1.

According to the method of the example in the present disclosure shownin FIG. 2, one of two directly-connected GR Restarters with the maximumID is taken as the master GR Restarter. The GR process of CE1, PE1, PE2and CE2 is shown in FIG. 5. FIG. 5 is a schematic diagram illustrating aGR process of four GR Restarters in the network shown in FIG. 1 inaccordance with an example of the present disclosure.

The master GR Restarters and slave GR Restarters are selected from anytwo directly-connected GR Restarters in the CE1, PE1, PE2 and CE2. TheCE1 directly connects with the PE1. Since the ID of the PE1 is largerthan that of the CE1, the PE1 is selected as the master GR Restarter ofthe CE1, and the CE1 is selected as the slave GR Restarter of the PE1.The PE1 directly connects with the PE2. Since the ID of the PE1 islarger than that of PE2, the PE1 is selected as the master GR Restarterof the PE2, and the PE2 is selected as the slave GR Restarter of thePE1. The CE2 directly connects with the PE2. Since the ID of the CE2 islarger than that of the PE2, the CE2 is selected as the master GRRestarter of the PE2, and the PE2 is selected as the slave GR Restarterof the CE2.

The PE1 is selected as the master GR Restarter of the CE1 and PE2. Afterreceiving routes from normal neighbor network device, i.e. the PE3 butnot its slave GR Restarters, i.e. the CE1 and PE2, the PE1 calculatesthe selected routes. Since complete routes from CE1 and PE2 (i.e. atleast two slave GR Restarters) are not received, the PE1 determines thatthe slave GR Restarter aggregation is {CE1, PE2}. The selected routesare denoted as incomplete routes, and sent to all the slave GRRestarters, i.e. the CE1 and PE2 in the slave GR Restarter aggregation.

CE2 is selected as the master GR Restarter of the PE2. The CE2 has noother neighbor network device except for the PE2, and it may beconsidered that complete routes of all neighbor network devices exceptfor the PE2 (i.e. at most one slave GR Restarter) are received.Therefore, the CE2 performs route selection to determine the selectedroutes. Since the CE2 receives complete routes of all neighbor networkdevices except for the PE2, the slave GR Restarter aggregation isdetermined as {PE2}. The selected routes are denoted as the completeroutes, and the complete routes are sent to the slave GR Restarter, i.e.PE2 in the slave GR Restarter aggregation.

The CE1 is selected as the slave GR Restarter of the PE1. The CE2 has noother neighbor network device except for the PE1, and it may beconsidered that complete routes of all the neighbor network devicesexcept for the PE1 are received. After receiving the routes which aredenoted as the incomplete routes and sent from the PE1, the CE1 performsroute selection to determine the selected routes, and determines thatthe master GR Restarter aggregation is {PE1}. Since the routes sent fromall the neighbor network devices except for the PE1 are received, theCE1 denotes the selected routes as the complete routes, and sends thecomplete routes to the PE1.

The PE2 is selected as the slave GR Restarter of the PE1 and CE2. Afterreceiving the routes sent from the PE3, and the routes which are denotedas the incomplete routes and sent from the PE1 and the routes which aredenoted as the complete routes and sent from the CE2, the PE2 performsroute selection to determine the selected routes. Since the completeroutes sent from the PE1 are not received, the master GR Restarteraggregation is determined as {PE1}, the selected routes are denoted asthe complete routes, and the complete routes are sent to the PE1.

After receiving routes which are denoted as complete routes and sentfrom the CE1 and PE2, the PE1 performs route selection to determine theselected routes. Since all the complete routes are received, the localFIB is updated with the selected routes, the selected routes are denotedas complete routes, and sent to all the neighbor network devices, i.e.CE1, PE2 and PE3, and the PE1 finishes the GR process.

After receiving the routes which are denoted as complete routes and sentfrom the PE1, the CE1 performs route selection to determine the selectedroutes. Since all the complete routes are received, the local FIB isupdated with the selected routes, the selected routes are denoted as thecomplete routes, and sent to the neighbor network device, i.e. the PE1,and the CE1 finishes the GR process.

After receiving the routes which are denoted as complete routes and sentfrom the PE1, the PE2 performs route selection to determine the selectedroutes. Since all the complete routes are received, the local FIB isupdated with the selected routes, the selected routes are denoted as thecomplete routes, the complete routes are sent to all the neighbornetwork devices, i.e. the PE1, PE3 and CE2, and the PE2 finishes the GRprocess.

After receiving the routes which are denoted as the complete routes andsent from the PE2, the CE2 performs route selection to determine theselected routes. Since all the complete routes are received, the localFIB is updated with the selected routes, which are denoted as completeroutes and sent to the neighbor network device, i.e. the PE2, the CE2finishes the GR process.

The GR process of the CE1, PE1, PE2 and CE2 is finished. Since each GRRestarter finishes the GR process after receiving all the completeroutes, the problem of traffic interruption can be reduced if notavoided. If the route information exchange is performed with all slaveGR Restarter(s) or master GR Restarter(s) for multiple times, the GRprocess of the CE1, PE1, PE2 and CE2 is slightly different from theabove GR process. Referring to FIG. 6, FIG. 6 is a schematic diagramillustrating another GR process of four GR Restarters in the networkshown in FIG. 1 in accordance with an example of the present disclosure.The specific process is not repeated here.

FIGS. 3, 4, 5 and 6 respectively show the GR process when the BGP oftwo, three and four network devices is restarted. In practice, the GRprocess of each GR Restarter may be different according to the number ofthe network devices, the BGP of which is restarted, and the specificsituation of the selected master and slave GR Restarters, based on theabove implementation principles.

The above describes several examples of the BGP GR method in detail.This disclosure further provides a BGP GR device, which may implementmultiple-points failure protection.

FIG. 7 is a schematic diagram illustrating structure of a device incapable of acting as a GR Restarter in a network with multiple GRRestarters that include a first GR Restarter and a corresponding secondGR Restarter accordance with a first example of the present disclosure.It should be understood that the device 710 is capable of acting as thefirst GR Restarter (master), or the second GR Restarter (slave), or boththe first GR Restarter and the second GR Restarter.

The GR Restarter in FIG. 7 includes: a memory 7120 storing instruction7122, and a processor 7110 to execute the instruction 7122 to perform GRfunctions. When acting as a first GR Restarter, the processor 7110 is toexecute instruction 7122 to:

after receiving routes from all normal neighbor network devices of thefirst GR Restarter, perform route selection to determine first selectedroutes, and send the first selected routes to the at least one second GRRestarter, and

after receiving routes from the at least one second GR Restarter,perform route selection to determine second selected routes, update alocal Forwarding Information Base (FIB) at the first GR Restarter withthe second selected routes, and send the second selected routes to allneighbor network devices of the first GR Restarter.

The first GR Restarter directly connects with the at least one second GRRestarter corresponding to the first GR Restarter.

Additionally or alternatively, when acting as a second GR Restarter, theprocessor 7110 is to execute instruction 7122 to:

after receiving routes from all neighbor network devices of the secondGR Restarter, perform route selection to determine third selectedroutes,

if complete routes from all the neighbor network devices of the secondGR Restarter are received, update a local FIB at the second GR Restarterwith the third selected routes; and

otherwise, send the third selected routes to the at least one first GRRestarter corresponding to second GR Restarter, perform route selectionto determine fourth selected routes until complete routes from allneighbor network devices of the second GR Restarter are received, updatethe local FIB with the fourth selected routes, and send the fourthselected routes to all the neighbor network devices of the second GRRestarter.

The second GR Restarter directly connects with the at least one first GRRestarters corresponding to the second GR Restarter.

Although the selected routes determined by the slave (second) GRRestarter have been referred to as the ‘third’ and ‘fourth’ selectedroutes, it should be understood that the terms ‘third’ and ‘fourth’ aremerely used to improve clarity and distinguish them from the ‘first’ and‘second’ selected routes determined by the master (first) GR restarter.This also applies to the use of ‘first’ and ‘second’ selected routesdetermined by the first GR Restarter.

FIG. 8 is a schematic diagram illustrating structure of a GR Restarterdevice in accordance with a second example of the present disclosure.The GR Restarter includes at least one of:

a first processing unit 810 (e.g. processor-in-memory etc.) and

a second processing unit 820 (e.g. processor-in-memory etc.).

As such, the GR Restarter may be capable of acting as a first GRRestarter (master), or a second GR Restarter (slave), or both (masterand slave). In cases were the GR Restarter is capable as acting as bothmaster and slave, it may be a master in relation or corresponding to atleast one other slave GR Restarter and a slave in relation orcorresponding to at least one other master GR Restarters, e.g. if thereare three GR Restarters identified as A, B and C. GR Restarter A may bea master corresponding to slave GR Restarter B and a slave correspondingto master GR Restarter C.

For example, referring to FIG. 4 again, if the ID of CE1 is greater thanthat of PE1 in a different scenario, PE1 is a slave GR Restartercorresponding to master GR Restarter CE1. In this case, PE1 is capableof acting as both as a master (corresponding to slave PE2) and a slave(corresponding to master CE1). It should also be understood that amaster may have more than one corresponding slave, and similarly, aslave may have more than one corresponding master. When the GR Restarteris capable of acting as both a master and slave, it will be appreciatedthat the functions of the first processing unit 810 and secondprocessing unit 820 may be combined into a single processing unit(similar to the example in FIG. 7).

Throughout this disclosure, although the terms ‘first’ and ‘second’ havebeen used to refer to the ‘processing units’ or ‘processor-in-memory’810 and 820, ‘memory’ 8122 and 8222, ‘processors’ 8110 and 8210 and‘instruction’ 8121, 8122, 8221 and 8222, it should be understood thatthe terms are merely labels for easy reference, and for distinguishingthem from each other. In practice, they may not necessarily be separateentities in separate devices. Similarly, the terms ‘first’, ‘second’,‘third’ and ‘fourth’ have been used to refer to notifications. Again,they are labels and should not be interpreted as implying any specificorder or sequence.

The first processor-in-memory 810 includes: a first processor 8110 and afirst memory 8120. The second memory processor 820 includes: a secondprocessor 8210 and a second memory 8220.

The first processor 8110 communicates with the first memory 8120,executes instructions saved in the first memory 8120, and the secondprocessor 8210 communicates with the second memory 8220, and executesinstructions saved in the second memory 8220.

The first memory 8120 includes: a first controlling instruction 8121 anda first receiving and sending instruction 8122, and the second memory8220 includes: a second controlling instruction 8221 and a secondreceiving and sending instruction 8222.

When the first processor-in-memory 810 is selected as a master GRRestarter (first GR Restarter) from multiple GR Restarters, the firstcontrolling instruction 8121 is executed to:

after the first receiving and sending instruction 8122 receives routesfrom all normal neighbor network devices of the first GR Restarter,perform route selection to determine first selected routes, and send afirst notification to the first receiving and sending instruction 8122,

after the first receiving and sending instruction 8122 sends the firstselected routes to the at least one second GR Restarter of the first GRRestarter according to the first notification and receiving routes fromthe at least one second GR Restarter of the first GR Restarter,performing route selection to determine second selected routes, update alocal FIB at the first GR Restarter with the second selected routes, andsend a second notification to the first receiving and sendinginstruction 8122.

The first receiving and sending instruction 8122 is executed to receivethe routes from all the neighbor network devices,

receive the first notification, send the first selected routes to the atleast one second GR Restarter of the first GR Restarter,

receive the second notification, and send the second selected routes toall the neighbor network devices of the first GR Restarter.

The first GR Restarter directly connects with the at least one second GRRestarter corresponding to the first GR Restarter. In one example, thenetwork may implement a Border Gateway Protocol (BGP) or any othersuitable routing protocol; and the first GR Restarter and each second GRRestarter are selected from the multiple GR Restarters as master GRRestarter and slave GR Restarter respectively. The multiple GRRestarters may form an aggregation.

When the second processor-in-memory 820 is selected as a slave GRRestarter from the multiple GR Restarters, the second controllinginstruction 8221 is executed to:

after routes from all neighbor network devices of the second GRRestarter are received, perform route selection to determine thirdselected routes, update a local FIB at the second GR Restarter with thethird selected routes if complete routes from all the neighbor networkdevices of the second GR Restarter are received,

otherwise, send a third notification to the second receiving and sendinginstruction, perform route selection to determine fourth selected routesuntil the second receiving and sending instruction receives completeroutes from all the neighbor network devices of the second GR Restarter,update the local FIB with the fourth selected routes, and send a fourthnotification to the second receiving and sending instruction.

The second receiving and sending instruction 8222 is executed to receivethe routes from the neighbor network devices of the second GR Restarter,

receive the third notification,

send the third selected routes to the at least one first GR Restartersof the second GR Restarter,

receive the fourth notification, and

send the fourth selected routes to all the neighbor network devices ofthe second GR Restarter.

The second GR Restarter directly connects with the at least one first GRRestarter corresponding to the second GR Restarter. The multiple GRRestarters may form an aggregation, in which each GR Restarter directlyconnects with at least one other GR Restarter in the aggregation. In oneexample, the network may implement a Border Gateway Protocol (BGP) orany other suitable routing protocol(s); and the second GR Restarter andeach first GR Restarter are selected from the multiple GR Restarters asslave GR Restarter and master GR Restarter respectively.

The first controlling instruction 8121 is further executed to determine,after the first selected routes are determined by performing routeselection, whether complete routes from all neighbor network devices ofthe first GR Restarter are received, and if yes, update the local FIBwith the first selected routes;

the first receiving and sending instruction is further executed to sendthe first selected routes to all second GR Restarters of the first GRRestarter.

The first receiving and sending instruction 8122 is further executed tosend the first selected routes to all second GR Restarters of the masterGR Restarter.

The second receiving and sending instruction 8222 is further executed tosend the third selected routes to all the master GR Restarters of theslave GR Restarter.

The first receiving and sending instruction 8122 is further executed toif complete routes from all the normal neighbor network devices of thefirst GR Restarter are received and if complete routes from at least twosecond GR Restarters of the first GR Restarter are not received,

-   -   denote the first selected routes as incomplete routes, and    -   send the first selected routes denoted as the incomplete routes        to all second GR Restarters of the first GR Restarter,

if the complete routes from all the normal neighbor network devices ofthe first GR Restarter are received and a complete route from at mostone second GR Restarter of the first GR Restarter is not received,

-   -   denote the first selected routes as the complete routes, and    -   send the first selected routes denoted as the complete routes to        the at most one second GR Restarter,    -   denote the first selected routes as the incomplete routes, and    -   send the first selected routes denoted as the incomplete routes        to any other second GR Restarter of the first GR Restarter,

if complete routes from all normal neighbor network devices of the firstGR Restarter are not received,

-   -   denote the first selected routes as the incomplete routes, and    -   send the first selected routes denoted as the incomplete routes        to all second GR Restarters of the first GR Restarter.

The second receiving and sending instruction 8222 is further executed toif the second GR Restarter receives complete routes from all normalneighbor network devices of the second GR Restarter and if an incompleteroute from one first GR Restarter of the second GR Restarter is notreceived, or

if complete routes from all the first GR Restarters are received butcomplete routes from at least two second GR Restarters of the second GRRestarter are not received,

-   -   denote the third selected routes as incomplete routes, and    -   send the third selected routes denoted as the incomplete routes        to all first GR Restarters of the second GR Restarter and any        other GR Restarter,

if the second GR Restarter receives complete routes from all the normalneighbor network devices of the second GR Restarter and does not receivea complete route from at most one second GR Restarter of the second GRRestarter,

-   -   denote the third selected routes as the complete routes, and    -   send the third selected routes denoted as the complete routes to        the at most one second GR Restarter,    -   denote the third selected routes as the incomplete routes, and    -   send the third selected routes denoted as the incomplete routes        to all first GR Restarters of the second GR Restarter and any        other GR Restarter.

The first controlling instruction 8121 is further executed to determine,after the first selected routes are determined, whether complete routesfrom all the neighbor network devices of the first GR Restarter arereceived, and if yes, update the local FIB with the first selectedroutes, and add, if the first GR Restarter does not receive a completeroute from a second GR Restarter, the second GR Restarter to a second GRRestarter aggregation.

The first receiving and sending instruction 8122 is further executed tosend the first selected routes to all second GR Restarters in the secondGR Restarter aggregation of the first GR Restarter.

The second controlling instruction 8221 is further executed to add thefirst GR Restarter to a first GR Restarter aggregation if a completeroute sent from the first GR Restarter is not received.

The second receiving and sending instruction 8222 is further executed tosend the third selected routes to all first GR Restarters in the firstGR Restarter aggregation of the second GR Restarter.

The first receiving and sending instruction 8122 is further executed to:

denote, if complete routes sent from all the normal neighbor networkdevices of the first GR Restarter are received and if there are at leasttwo second GR Restarters in the second GR Restarter aggregation, thefirst selected routes as incomplete routes, and send the first selectedroutes denoted as incomplete routes to all second GR Restarters in thesecond GR Restarter aggregation,

denote, if the complete routes sent from all the normal neighbor networkdevices of the first GR Restarter are received and there is at most onesecond GR Restarter in the second GR Restarter aggregation, the firstselected routes as the complete routes, and send the first selectedroutes denoted as the complete routes to the at most one second GRRestarter,

denote, if complete routes sent from all the normal neighbor networkdevices of the first GR Restarter are not received, the first selectedroutes as the incomplete routes, and send the first selected routesdenoted as the incomplete routes to all the second GR Restarters in thesecond GR Restarter aggregation.

The second receiving and sending instruction 8222 is further executedto:

if the second GR Restarter receives complete routes sent from all normalneighbor network devices of the second GR Restarter and if an incompleteroute sent from one first GR Restarter of the second GR Restarter is notreceived,

-   -   denote the third selected routes as the incomplete routes, and        send the third selected routes denoted as the incomplete routes        to the first GR Restarter,

if the second GR Restarter receives complete routes sent from all normalneighbor network devices of the second GR Restarter and does not receivea complete route sent from at most one second GR Restarter,

-   -   denote the third selected routes as complete routes, and send        the third selected routes denoted as complete routes to the at        most one second GR Restarter,    -   denote the third selected routes as incomplete routes, and send        the third selected routes denoted as incomplete routes to all        the first GR Restarters in the first GR Restarter aggregation.

The first receiving and sending instruction 8122 is further executed todenote the first selected routes as the complete routes by generating afirst EOR message which does not carry a Partial flag, and denote thefirst selected routes as the incomplete routes by generating a secondEOR message carrying the Partial flag.

The second receiving and sending instruction 8222 is further executed todenote the third selected routes as the complete routes by generating athird EOR message which does not carry the Partial flag, and denote thethird selected routes as the incomplete routes by generating a fourthEOR message carrying the Partial flag.

The above are examples of the disclosure along with some variations. Theterms, descriptions and figures used herein are set forth by way ofillustration only and are not meant as limitations. Many variations arepossible within the scope of the disclosure, which is intended to bedefined by the following claims—and their equivalents—in which all termsare meant in their broadest reasonable sense unless otherwise indicated.

What is claimed is:
 1. A Graceful Restart (GR) method for a network withmultiple GR Restarters that include a first GR Restarter and at leastone second GR Restarter corresponding to the first GR Restarter, themethod comprising the first GR Restarter: after receiving information onroutes from all normal neighbor network devices of the first GRRestarter, performing route selection to determine first selectedroutes, and sending information on the first selected routes to the atleast one second GR Restarter, after receiving information on routesfrom the at least one second GR Restarter, performing route selection todetermine second selected routes, updating a local ForwardingInformation Base (FIB) at the first GR Restarter with information on thesecond selected routes, and sending the information on the secondselected routes to all neighbor network devices of the first GRRestarter; wherein the first GR Restarter directly connects with the atleast one second GR Restarter corresponding to the first GR Restarter.2. The method according to claim 1, wherein the network implements aBorder Gateway Protocol (BGP), and wherein the first GR Restarter andeach second GR Restarter are selected from the multiple GR Restarters asmaster GR Restarter and slave GR Restarter respectively and the multipleGR Restarters form an aggregation.
 3. The method according to claim 1,wherein after the first selected routes are determined, the methodfurther comprises: determining whether complete routes from all neighbornetwork devices of the first GR Restarter are received, and if yes,updating the local FIB with the information on the first selectedroutes; and sending the information on the first selected routes to allsecond GR Restarters corresponding to the first GR Restarter.
 4. Themethod according to claim 3, wherein sending the information on thefirst selected routes to all second GR Restarters of the first GRRestarter comprises: if information on complete routes from all normalneighbor network devices of the first GR Restarter are received andinformation on complete routes from at least two second GR Restarterscorresponding to first GR Restarter are not received, denoting the firstselected routes as incomplete routes, and sending the information on thefirst selected routes denoted as the incomplete routes to all second GRRestarters corresponding to first GR Restarter, if the information onthe complete routes from all normal neighbor network devices of thefirst GR Restarter are received but information on a complete route fromat most one second GR Restarter corresponding to first GR Restarter isnot received, denoting the first selected routes as the complete routes,and sending the information on the first selected routes denoted as thecomplete routes to the at most one second GR Restarter, denoting thefirst selected routes as the incomplete routes, and sending theinformation on the first selected routes denoted as the incompleteroutes to any other second GR Restarter corresponding to first GRRestarter, if the information on complete routes from all normalneighbor network devices of the first GR Restarter are not received,denoting the first selected routes as the incomplete routes, and sendingthe information on the first selected routes denoted as the incompleteroutes to all second GR Restarters corresponding to first GR Restarter.5. The method according to claim 1, wherein after the first selectedroutes are determined, the method further comprises: determining whetherinformation on complete routes from all neighbor network devices of thefirst GR Restarter are received, and if yes, updating the local FIB withthe first selected routes; if the first GR Restarter does not receiveinformation on a complete route from a second GR Restarter, adding thesecond GR Restarter to a second GR Restarter aggregation; and sendingthe information on the first selected routes to the at least one secondGR Restarter corresponding to first GR Restarter, wherein sending theinformation on the first selected routes further comprises sending theinformation on the first selected routes to all second GR Restarters inthe second GR Restarter aggregation of the first GR Restarter.
 6. Themethod according to claim 4, wherein denoting the first selected routesas the complete routes comprises: generating a first End-of-RIB (EOR)message which does not carry a Partial flag; and denoting the firstselected routes as the incomplete routes, wherein denoting the firstselected routes as the incomplete routes comprises generating a secondEOR message carrying the Partial flag.
 7. The method according to claim2, wherein each of the at least one first GR Restarter corresponding tothe second GR Restarter is selected from one of two directly-connectedGR Restarters in the multiple GR Restarters with the maximum or minimumID.
 8. A Graceful Restart (GR) method for a network with multiple GRRestarters that include a second GR Restarter and at least one first GRRestarter corresponding to the second GR Restarter, the methodcomprising the second GR Restarter: after receiving information onroutes from all neighbor network devices of the second GR Restarter,performing route selection to determine first selected routes, ifinformation on complete routes from all the neighbor network devices ofthe second GR Restarter are received, updating a local FIB at the secondGR Restarter with information on the first selected routes; otherwise,sending information on the first selected routes to the at least onefirst GR Restarter corresponding to second GR Restarter, performingroute selection to determine second selected routes until information oncomplete routes from all neighbor network devices of the second GRRestarter are received, updating the local FIB with information on thesecond selected routes, and sending the information on the secondselected routes to all the neighbor network devices of the second GRRestarter; wherein the second GR Restarter directly connects with the atleast one first GR Restarters corresponding to the second GR Restarter.9. The method according to claim 8, wherein the network implements aBorder Gateway Protocol (BGP), and wherein the second GR Restarter andeach first GR Restarter are selected from the multiple GR Restarters asslave GR Restarter and master GR Restarter respectively and the multipleGR Restarters form an aggregation.
 10. The method according to claim 8,wherein sending the information on the first selected routes to the atleast one first GR Restarter corresponding to the second GR Restartercomprises sending the information on the first selected routes to allfirst GR Restarters corresponding to second GR Restarter.
 11. The methodaccording to claim 10, wherein sending the information on the firstselected routes to all first GR Restarters corresponding to second GRRestarter comprises: if the second GR Restarter receives information oncomplete routes from all normal neighbor network devices of the secondGR Restarter and if information on an incomplete route from one first GRRestarter corresponding to second GR Restarter is not received; or ifinformation on complete routes from all first GR Restarters are receivedbut information on complete routes from at least two second GRRestarters are not received, denoting the first selected routes asincomplete routes, and sending the information on the first selectedroutes denoted as the incomplete routes to all first GR Restarterscorresponding to second GR Restarter and any other GR Restarter, if thesecond GR Restarter receives the information on complete routes from allnormal neighbor network devices of the second GR Restarter and does notreceive information on a complete route from at most one master GRRestarter corresponding to second GR Restarter, denoting the firstselected routes as the complete routes, and sending the information onthe first selected routes denoted as the complete routes to the at mostone second GR Restarter, denoting the first selected routes as theincomplete routes, and sending the information on the first selectedroutes denoted as the incomplete routes to all first GR Restarterscorresponding to second GR Restarter and any other GR Restarter.
 12. Themethod according to claim 8, wherein the method further comprises:adding, if information on a complete route from a first GR Restarter isnot received, the first GR Restarter to a first GR Restarteraggregation; and sending the information on the first selected routes tothe first GR Restarters corresponding to second GR Restarter, whereinsending the information on the first selected routes comprises sendingthe information on the first selected routes to all first GR Restartersin the first GR Restarter aggregation of the second GR Restarter. 13.The method according to claim 11, wherein denoting the first selectedroutes as the complete routes comprises: generating a first End-of-RIB(EOR) message which does not carry the Partial flag; and denoting thefirst selected routes as the incomplete routes, wherein denoting thefirst selected routes as the incomplete routes comprises generating asecond EOR message carrying the Partial flag.
 14. A device capable ofacting as a first Graceful Restart (GR) Restarter in a network withmultiple GR Restarters that include the first GR Restarter and at leastone second GR Restarter corresponding to the first GR Restarter, thedevice comprising: a memory and a processor to execute instructions inthe memory to: after receiving information on routes from all normalneighbor network devices of the first GR Restarter, perform routeselection to determine first selected routes, and send information onthe first selected routes to the at least one second GR Restarter, afterreceiving information on routes from the at least one second GRRestarter, perform route selection to determine second selected routes,update a local Forwarding Information Base (FIB) at the first GRRestarter with information on the second selected routes, and send theinformation on the second selected routes to all neighbor networkdevices of the first GR Restarter; wherein the first GR Restarterdirectly connects with the at least one second GR Restartercorresponding to the first GR Restarter.
 15. The device according toclaim 14, wherein the network implements a Border Gateway Protocol(BGP), and wherein the first GR Restarter and each second GR Restarterare selected from the multiple GR Restarters as master GR Restarter andslave GR Restarter respectively, and the multiple GR Restarters form anaggregation.
 16. The device according to claim 14, wherein the processorfurther executes instructions to: determine, after the first selectedroutes are determined, whether information on complete routes from allneighbor network devices of the first GR Restarter are received, and ifyes, update the local FIB with the information on the first selectedroutes; and send the information on the first selected routes to allsecond GR Restarters corresponding to first GR Restarter.
 17. The deviceaccording to claim 16, wherein the processor further executesinstructions to: if information on complete routes from all the normalneighbor network devices of the first GR Restarter are received and ifinformation on complete routes from at least two second GR Restarterscorresponding to first GR Restarter are not received, denote the firstselected routes as incomplete routes, and send the information on thefirst selected routes denoted as the incomplete routes to all second GRRestarters corresponding to first GR Restarter, if the information onthe complete routes from all the normal neighbor network devices of thefirst GR Restarter are received and information on a complete route fromat most one second GR Restarter corresponding to first GR Restarter isnot received, denote the first selected routes as the complete routes,and send the information on the first selected routes denoted as thecomplete routes to the at most one second GR Restarter, denote the firstselected routes as the incomplete routes, and send the information onthe first selected routes denoted as the incomplete routes to any othersecond GR Restarter corresponding to first GR Restarter, if theinformation on complete routes from all normal neighbor network devicesof the first GR Restarter are not received, denote the first selectedroutes as the incomplete routes, and send the information on the firstselected routes denoted as the incomplete routes to all second GRRestarters corresponding to first GR Restarter.
 18. The device accordingto claim 14, wherein the processor further executes instructions to:determine, after the first selected routes are determined, whetherinformation on complete routes from all the neighbor network devices ofthe first GR Restarter are received, and if yes, update the local FIBwith the information on the first selected routes, and add, if the firstGR Restarter does not receive information on a complete route from asecond GR Restarter, the second GR Restarter to a second GR Restarteraggregation; and send the information on the first selected routes toall second GR Restarters in the second GR Restarter aggregation of thefirst GR Restarter.
 19. The device according to claim 17, wherein theprocessor further executes instructions to: denote the first selectedroutes as the complete routes by generating a first EOR message whichdoes not carry a Partial flag, and denote the first selected routes asthe incomplete routes by generating a second EOR message carrying thePartial flag.
 20. A device capable of acting as a second GracefulRestart (GR) Restarter for a network with multiple GR Restarters thatinclude the second GR Restarter and at least one first GR Restartercorresponding to the second GR Restarter, the device comprising: amemory, and a processor to execute instructions in the memory to: afterreceiving information on routes from all neighbor network devices of thesecond GR Restarter, perform route selection to determine first selectedroutes, if information on complete routes from all the neighbor networkdevices of the second GR Restarter are received, update a local FIB atthe second GR Restarter with information on the first selected routes;otherwise, send information on the first selected routes to the at leastone first GR Restarter corresponding to second GR Restarter, performroute selection to determine second selected routes until information oncomplete routes from all neighbor network devices of the second GRRestarter are received, update the local FIB with information on thesecond selected routes, and send information on the second selectedroutes to all the neighbor network devices of the second GR Restarter;wherein the second GR Restarter directly connects with the at least onefirst GR Restarters corresponding to the second GR Restarter.
 21. Thedevice according to claim 20, wherein the network implements a BorderGateway Protocol (BGP), and wherein the second GR Restarter and eachfirst GR Restarter are selected from the multiple GR Restarters as slaveGR Restarter and master GR Restarter respectively, and the multiple GRRestarters form an aggregation.
 22. The device according to claim 20,wherein the processor further executes instructions to send theinformation on the first selected routes to all first GR Restarterscorresponding to the second GR Restarter.
 23. The device according toclaim 22, wherein the processor further executes instructions to: if thesecond GR Restarter receives information on complete routes from allnormal neighbor network devices of the second GR Restarter and ifinformation on an incomplete route from one first GR Restartercorresponding to second GR Restarter is not received, or if informationon complete routes from all first GR Restarters are received butinformation on complete routes from at least two second GR Restartersare not received, denote the first selected routes as incomplete routes,and send information on the first selected routes denoted as theincomplete routes to all first GR Restarters corresponding to second GRRestarter and any other GR Restarter, if the second GR Restarterreceives the information on complete routes from all the normal neighbornetwork devices of the second GR Restarter and does not receiveinformation on a complete route from at most one second GR Restartercorresponding to second GR Restarter, denote the first selected routesas the complete routes, and send the information on the first selectedroutes denoted as the complete routes to the at most one second GRRestarter, denote the first selected routes as the incomplete routes,and send the information on the first selected routes denoted as theincomplete routes to all first GR Restarters corresponding to second GRRestarter and any other GR Restarter.
 24. The device according to claim20, wherein the processor further executes instructions to: ifinformation on a complete route from a first GR Restarter is notreceived, add the first GR Restarter to a first GR Restarteraggregation; and send the information on the first selected routes toall first GR Restarters in the first GR Restarter aggregationcorresponding to second GR Restarter.
 25. The device according to claim23, wherein the processor further executes instructions to denote thefirst selected routes as the complete routes by generating a first EORmessage which does not carry the Partial flag, and denote the firstselected routes as the incomplete routes by generating a second EORmessage carrying the Partial flag.